Three standard techniques used in parallel processing architectures: these include pipelining, use of multiple simple processing elements, and multi-processing. Pipelines are used to achieve time parallelism, where the independent steps in multi-cycle operations are executed in parallel, thereby improving performance. An array of Processing Elements (PEs), termed an array processor, achieves a physical parallelism through operating the multiple elements synchronously in parallel. A multi-processing system achieves an asynchronous parallelism with multiple communicating processes executing on independent systems. This invention is concerned with the pipeline control of multiple processing elements connected together in some topology.